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ABSTRACT 

The  use  of  web-based  tools  for  viewing  and  sharing 
imagery  has  expanded  significantly  over  the  last  few 
years.  Web  services  such  as  Google  Earth  have  whet  the 
public’s  appetite  for  georegistered  and  georectified 
imagery.  While  there  is  a  large  amount  of  registered 
imagery  available  for  display  on  these  Servers,  much  of 
the  data  is  old  and  outdated,  some  by  many  years. 
NAVSYS  has  developed  a  GP S/inertial/video  product 
(GI-Eye)  that  can  be  used  to  collect  georegistered  imagery 
in  real-time.  We  have  combined  this  with  a 
GeoReferenced  Information  Manager  (GRIM)  Server  that 
allows  this  imagery  to  be  converted  to  a  georectified 
image.  In  this  paper,  we  describe  the  GI-Eye  and  GRIM 
products  and  the  auto-mosaicing  capability  that  they 
provide.  We  also  include  examples  of  mosaic  imagery 
that  was  generated  during  flight  tests  of  the  system. 


INTRODUCTION 

The  use  of  georeferenced  imagery  across  the  Internet  is 
becoming  prevalent  thanks  to  the  development  of  web- 
based  location  servers  such  as  Google  Earth,  TerraServer 
and  Yahoo  Local.  One  of  the  most  frequently  asked 
questions  for  these  services  is  '"How  old  is  your  data?'' 
The  answer  posted  by  Google  to  this  question  is  "'Google 
Earth  acquires  the  best  imagery  available,  most  of  which 
is  approximately  one  to  three  years  old.  Users  of  these 
services  are  continually  asking  for  more  timely,  high 
resolution  data.  The  military  also  has  a  similar  need  to 
provide  near  real-time  situational  awareness  to  ground 
troops.  This  is  currently  provided  using  streaming  video, 
but  this  does  not  provide  the  same  geospatial  awareness 
that  the  3D  Google  Earth-like  tools  provide.  Civil 
agencies  such  firefighters,  search  and  rescue  teams,  law 
enforcement,  911  emergency  operations,  border  patrol 
operations,  traffic  monitoring  systems,  and  geological 
survey  crews  could  also  benefit  from  a  near  real-time 
web-based  geospatial  visualization  capability. 

To  address  this  need  for  real-time  geospatial  awareness, 
NAVSYS  has  developed  the  GI-Eye  product  which 
provides  the  capability  to  generate  precision  mensurated 
imagery  directly  on  the  aircraft  collecting  the  data.  The 
GI-Eye  system  integrates  GPS,  inertial  and  digital  camera 
data  to  provide  autonomous  registration  capability  for 
imagery  without  requiring  access  to  any  Ground  Control 
Points  (GCPs).  This  provides  real-time,  high  quality 
registered  imagery  at  a  1-Hz  rate.  The  GI-Eye  system  has 
been  combined  with  an  Enterprise  Server,  termed  the 
GeoReferenced  Information  Manager  (GRIM)  that  uses 
this  mensurated  imagery  to  auto-generate  mosaics  as  the 
data  is  being  collected.  With  this  approach,  a  near  real¬ 
time  geospatial  view  of  the  environment  can  be  generated 
in  a  format  that  can  be  viewed  using  the  current  web- 
based  geospatial  visualization  tools. 

GI-EYE  SYSTEM  DESIGN 

The  GI-Eye  product  is  offered  to  sensor  manufacturers 
and  systems  integrators  to  provide  an  embedded  precision 
autoregistration  capability  for  electro-optic  (EO),  infra¬ 
red  (IR)  or  other  focal  plane  array  (FPA)  type  sensors. 


Proceedings  of  ION  GNSS  2006,  Fort  Worth,  Texas,  September  2006 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 

1.  REPORT  DATE 

SEP  2006  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-09-2006  to  00-09-2006 

4.  TITLE  AND  SUBTITLE 

Real-Time  Web-based  Image  Distribution  using  an  Airborne 

GPS/Inertial  Image  Server 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

NAVSYS  Corporation, 14960  Woodcarver  Road, Colorado 
Springs,CO,80921 

8.  PEREORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIEICATION  OE:  17.  LIMITATION  OE 

APl9!TR  apt 

18.  NUMBER  19a.  NAME  OE 

/TTh  PAPIhQ  DTnQP/TMQTTiT  In  PlnDQ/TM 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

unclassified  unclassified  unclassified 

8 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


This  product  has  been  integrated  with  a  variety  of 
different  digital  cameras  and  sensors.  A  ground-based 
system  being  used  for  tactical  surveying  and  targeting  by 
the  National  Geospatial-Intelligence  Agency  (NGA)  is 
shown  in  Figure  1.  GI-Eye  is  also  being  used  by  FLIR  in 
their  S tar S AFIRE®  III  product  (Figure  2)  where  it 
provides  a  GeoPointing^^^  capability  to  stabilize  their  high 
resolution  imaging  to  a  point  on  the  ground  selected  by 
the  operator. 


Figure  1  NGA  Tactical  Surveying  and  Targeting 
System  (TS2) 


Figure  2  FLIR  StarSAFIRE 

The  GI-Eye  product  provides  the  capability  to  precisely 
time  mark  each  camera  image  and  uses  NAVSYS’ 
proprietary  InterNav  kinematic  alignment  algorithm^"^^  to 
measure  the  precise  position  and  attitude  of  the  camera 
using  the  GPS  and  inertial  sensor  data.  The  GI-Eye 
system  auto-registration  capability  provides  the  location 
and  pointing  angle  of  the  sensor  with  each  image  and  also 
sensor  calibration  data  from  which  the  coordinates  of  each 
pixel  in  the  image  can  be  derived.  This  information  can  be 
used  with  a  Digital  Elevation  Model  (DEM)  to  extract  the 
individual  pixel  coordinates  of  each  image  (Figure  3).  It 
can  also  be  used  to  derive  a  3D  DEM  from  multiple 
images  through  photogrammetry.  With  the  precision  GI- 
Eye  meta-data,  there  is  no  need  for  any  GCPs  to  be  used 
for  image  registration.  A  self-calibration  capability  has 
been  designed  into  the  GI-Eye  system  that  allows  for 


estimation  of  camera  misalignment,  focal  length  and  lens 
distortion  parameters.  Figure  4  shows  the  improvement  in 
the  targeting  precision  that  results  from  applying  these 
calibration  parameters  to  a  test  set  of  data  collected 
against  a  known  reference  point. 


Figure  3  GI-Eye  Position  and  Attitude  Image 
Registration 

NAVSYS’  technology  has  resulted  in  improved  position 
and  attitude  accuracy  using  relatively  inexpensive  inertial 
components.  This  allows  georegistration  accuracies  of  1-2 
meters  to  be  provided  when  flying  at  an  altitude  of  1,000 
feet  without  requiring  the  use  of  any  ground  truth^^^. 


Figure  4  GI-Eye  Calibration  Results 

UNMANNED  AIR  VEHICLE  (UAV)  PAYLOAD 

The  GI-Eye  has  also  been  configured  for  use  in  a  UAV 
payload.  Our  approach  is  to  use  a  modular  design  that 
enables  sensor  upgrades  over  time  with  only  software 
configuration  changes.  This  allows  us  to  take  advantage 
quickly  of  advances  in  sensor  and  inertial  subsystems. 
The  GI-Eye  system  components  are  shown  in  Figure  5. 
From  left  to  right,  the  figure  shows  the  digital  camera. 


power  converter  board,  IMU  interface  board,  single  board 
computer,  and  the  BAE  Multisensor  Inertial  Measurement 
Unit  (MIMU).  The  hard  drive  is  positioned  under  the 
single  board  computer.  In  Figure  6  a  drawing  is  shown  of 
these  components  assembled  in  the  ARES  UAV  payload. 
ARES  is  a  small  UAV  developed  by  the  Research  and 
Engineering  Center  for  Unmanned  Vehicles  (RECUV)  at 
the  University  of  Colorado  at  Boulder  for  use  in  flight 
testing  advanced  unmanned  air  system  concepts’^^l  The 
camera  and  IMU  can  be  either  fixed  in  the  aircraft  or 
mounted  inside  a  gimbal  to  allow  geopointing  to  targets 
of  interest.  The  individual  components  are  described 
below.  The  size,  weight  and  power  specifications  for  the 
GI-Eye  UAV  payload  are  included  in  Table  1. 


Figure  5  GI-Eye  UAV  Payload  Components 


GPS  Receiver  and  Antenna  A  NovAtel  GPS  receiver  is 
being  used  for  the  GI-Eye.  For  high  accuracy 
applications,  we  can  use  their  L1/L2  observations  to 
perform  kinematic  processing.  The  default  mode  is  to  use 
either  Wide  Area  Augmentation  System  (WAAS) 
corrections  or  Differential  GPS  (DGPS)  corrections  from 
the  Ground  Station  to  provide  the  sensor  coordinates. 

IMU  and  NAVSYS  Interface  Module  (NIM)  InterNav 
supports  a  number  of  different  IMUs.  For  the  UAV  GI- 
Eye  configuration,  we  selected  the  BAE  Multisensor 
Inertial  Measurement  Unit.^^^  The  NIM  is  a  customized 
board  developed  by  NAVSYS  that  provides  an  adaptable 
interface  between  the  IMU  and  single  board  computer 
through  an  RS232  or  USB  port.  It  also  time  synchronizes 
the  IMU  and  GPS  data. 

Digital  Camera  The  digital  camera  uses  a  high  resolution, 
color  CMOS  video  sensor  with  a  maximum  resolution  of 
1280  X  1024  at  15  frames  per  second.  The  standard  USB 
2.0  interface  is  capable  of  transferring  data  at  480  Mbps 
and  is  integrated  with  a  built-in  frame  buffer  to  prevent 
data  loss.  The  camera  is  plug-n-play  and  supports 
Windows  2000/XP.  The  camera  uses  a  C  mount  lens  with 
a  Vi'  optical  format.  A  variety  of  lens  options  are  readily 
available  from  a  number  of  manufactures  ranging  from 


fixed-focus  6  mm  and  smaller  to  75  mm  and  zoom  lens. 
Flexibility  in  lens  selection  is  a  key  aspect  of  the  payload 
design  because  the  lens  characteristics  define  a  number 
performance  attributes  of  the  system.  In  operational 
scenarios  where  the  long  distance  target  identification  is 
important,  a  lens  with  a  narrow  field  of  view  is  required. 
If  a  significant  overlap  between  images  is  important  for 
image  tracking  or  mosaic  generation,  a  lens  with  a  wider 
field  of  view  may  be  required.  Therefore,  by  simply 
swapping  lenses  on  the  digital  camera,  different  mission 
objects  can  be  accomplished  with  minimal  effort. 

Single  Board  Computer  (SBC)  The  SBC  is  the  core 
processing  and  control  element  of  the  payload.  It  is  a 
Pentium-M  CPU-based  computer  in  a  PC/I  04-Plus  form 
factor,  which  provides  a  relatively  high  performance 
processing  platform  that  is  light  weight  with  little  power 
consumption.  The  integrated  USB  2.0,  RS232,  and 
Ethernet  ports  provide  a  flexible  means  of  interfacing  the 
SBC  with  the  payload  components  while  the  video  and 
USB  interfaces  create  a  convenient  development  and 
debugging  environment. 

Power  Supply  The  GI-Eye  system  operates  from  a  12 
VDC  power  supply.  Total  power  consumption  is  35.4 
watts  nominal  and  56  watts  peak.  A  DC/DC  power  supply 
is  included  in  the  package  to  power  the  subsystem 
components. 


Figure  6  GI-Eye  UAV  Mechanical  Assembly 


Figure  7  ARES  Unmanned  Air  Vehicle 


Table  1  Payload  Weight  and  Power  Summary 


Description 

Weight 

(g) 

Power 

(W) 

GPS  Antenna 

145 

0.125 

GPS  Engine 

56 

1.600 

IMU 

539 

16.850 

Digital  Camera 

176 

1.500 

Lens 

55 

0.000 

Interface  Board 

25 

0.300 

SBC 

250 

13.000 

HDD 

115 

2.000 

Power  Supply 

105 

Cable  Budget 

200 

Camera/IMU  Platform 

300 

1965 

35.375 

4.3 

lbs 

GEOREFERENCED  INFORMATION  MANAGER 
(GRIM) 

The  companion  product  to  the  GI-Eye  is  the 
GeoReferenced  Information  Manager.  GRIM  is 
developed  based  on  an  Oracle  Application  Server 
architecture.  GRIM  provides  tools  to  synchronize  data 
between  the  UAV  payload  and  the  GRIM  server  and  to 


provide  management  and  intelligent  search  and  retrieval 
of  the  image  data.  GRIM  leverages  the  GI-Eye  meta-data 
which  provides  the  precise  location  and  attitude  of  the 
sensor  images  to  simplify  and  streamline  feature 
extraction  from  the  images.  The  precise  sensor  meta-data 
also  eliminates  the  need  for  expensive  and  time 
consuming  image  processing  for  generating  products  such 
as  mosaics  or  digital  elevation  models.  The  interfaces 
between  the  GRIM  server  and  the  airborne  GI-Eye 
payload  are  shown  in  Figure  8. 

GRIM  adopts  an  Enterprise  Architecture  to  manage  the 
GI-Eye  information.  The  Oracle  database  connection  is 
used  to  synchronize  the  GI-Eye  metadata  and  image 
thumbnails  between  the  Oracle  Express  database  in  the 
GI-Eye  and  the  GRIM  database.  This  allows  GRIM  to  use 
Web  Services  to  prioritize  the  full  image  data  for  transfer 
across  the  downlink  using  FTP  (“smart  push”).  Tools  are 
included  in  the  GRIM  server  to  allow  for  location-based 
search  and  retrieval  of  the  GI-Eye  sensor  data  (“smart 
pull”).  Users  can  access  the  GRIM  viewing  and  targeting 
tools  through  a  web  browser  as  shown  in  Figure  9  for 
real-time  access  to  the  georegistered  imagery.  The  user 
can  also  use  GRIM  to  automatically  select  multiple 
images  over  a  target.  A  web-based  user  interface  is 
provided  to  allow  the  user  to  select  the  target  pixels  in 
each  image.  The  GRIM  server  will  then  automatically 
calculate  the  3D  target  coordinates  and  error  ellipse  from 
this  data  and  display  it  to  the  user  (Figure  10). 
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Figure  8  GI-Eye  and  GRIM  Architecture 


Figure  9  GRIM  Web  Browser  and  Image  Manager 

AUTO-MOSAIC  GENERATION 

The  GRIM  product  has  been  designed  to  optionally 
include  an  auto-mosaicing  function  that  can  take  the 
down-linked  GI-Eye  images  and  create  a  mosaic  in  near 
real-time.  This  allows  the  downlinked  imagery  to  be 
displayed  as  a  registered  “overhead”  image  using  existing 
tools  such  as  FalconView  or  Google  Earth. 

The  process  of  mosaicing  is  joining  together  two  or  more 
overlapping  images  to  form  a  continuous  composite 
image.  The  normal  method  of  generating  a  mosaic  is 
illustrated  in  Figure  11  and  requires  intensive  image 
processing  to  collect  ground  control  points  and  compute  a 


Target 

Position 

Estimation 


Figure  10  GRIM  Precision  Targeting  Calculation 


math  model  for  use  in  orthorectifying  the  images  and 
creating  the  mosaic.  With  the  GI-Eye  meta-data,  the  math 
model  for  each  image  can  be  explicitly  calculated 
eliminating  these  time  consuming  steps. 

Figure  12  shows  the  Auto-Mosaicing  process  that  has 
been  developed  to  be  run  on  the  GRIM  Server  using  PCI 
Geomatics’  Production  Workflow  products^^^.  This  uses  a 
recursive  process  to  orthorectify  images  from  the  GRIM 
database  and  uses  these  to  update  the  mosaic  image.  This 
process  can  be  run  in  near  real-time  allowing  an  overhead 
view  to  be  generated  potentially  within  minutes  of  the 
images  being  collected.  Examples  of  a  mosaic  generated 
from  GI-Eye  using  this  product  are  shown  in  Figure  13. 


Figure  11  Mosaic  generation  using  PCI  Geomatics  Tools^^^ 
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Figure  12  Auto-Mosaic  Generation  using  PCI  Geomatics  Tools  with  GI-Eye  meta-data 
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Figure  13  Auto-Mosaic  Generated  from  GI-Eye  Flight  Test 
GRIM  FEATURES  AND  BENEFITS 

The  GRIM  system  has  been  developed  based  on  an 
Oracle  Application  Server  which  incorporates  spatial 
processing  in  accordance  with  the  open  geospatial 
standards  developed  by  the  Open  Geospatial  Consortium 
(OGC).  This  allows  the  GRIM  imagery  products  to  be 
integrated  with  Web  geospatial  tools  that  are  also  OGC 
compliant.  An  example  of  an  application  that  is 
interoperable  with  the  GRIM  server  is  eSpatiaTs 
iSMART  product^^^.  As  shown  in  Figure  14,  this  allows 
the  GRIM  developed  mosaics  to  be  viewed  in  real-time 
with  other  data  overlays  in  user  customized  web  pages 
using  only  a  web  browser.  The  iSMART  product  also 
allows  geospatial  data  to  be  published  back  into  the 
GRIM  server  from  user  entered  data  on  the  web  page. 

This  allows  real-time  updates  to  be  made  for  feature  data 
and  real  time  display  of  registered  airborne  imagery 
collected  from  the  GI-Eye  system. 


Figure  14  eSpatiaFs  iSMART  product 

A  summary  of  the  features  and  benefits  provided  by  the 
GRIM  tools  set  is  included  below. 


Table  2  Summary  of  GRIM  Features  and  Benefits 


Feature 

Conventional  UAV 
Approach 

GRIM  Approach 

Imagery 

Downlink 

Streaming  video 
imagery 
downlinked. 
Compression 
needed  due  to 
bandwidth 
constraints 
compromises  on 
image  quality. 

High  resolution  still- 
images  downlinked. 
Real-time  low 
resolution  display 
provided  for  User 
Interface. 

Search 

and 

Retrieval 

Fast-rewind 
capability  provided 
for  review  of  past 
data  -  searches 
through  time  only. 

Intelligent  search 
and  retrieval  of 
imagery  data  based 
on  time  window 
and/or  target 
coordinates. 

Visual 

Display 

Streaming  imagery 
-  requires  trained 
operator  to  interpret 
“bird’s  eye”  view 

Real-time  Mosaic 
generation  - 
minimizes  operator 
training  by  allowing 
overlay  on  existing 
tools  such  as 
FalconView  or 

Google  Earth 

Targeting 

Accuracy  of 
targeting  is 
generally  poor  due 
to  low  resolution 
metadata  and 
reduced  quality 
imagery 

High  accuracy 
targeting  based  on 
real-time 
georeferenced 

Mosaic  (point  and 
click  user  interface) 

SUGGESTED  CONCEPT  OF  OPERATIONS  FOR 
NEAR  REAL-TIME  MOSAIC  GENERATION 


Using  the  GI-Eye  and  GRIM  technology,  it  would  be 
possible  to  generate  in  near  real-time  a  continually 
updated  registered  mosaic  with  an  overhead  view  of  the 
area  covered.  A  potential  concept  of  operations  for  this 
capability  is  illustrated  in  Figure  15.  A  network  of  low- 
cost  persistent  UAVs  with  GI-Eye  payloads  could  be  used 
to  collect  real-time  registered  imagery  which  is  then 
downlinked  to  a  central  GRIM  server.  The  imagery  can  be 
processed  in  near  real-time  creating  a  registered  overhead 
view  in  a  format  that  can  be  used  by  location  servers  to 
provide  a  current  view  of  the  area  covered  by  the  UAVs. 

CONCLUSION 

Near  real-time  dissemination  of  georeferenced  imagery  by 
a  location  server  can  benefit  a  number  of  military  and 
civilian  applications.  The  military  would  benefit  from 
having  access  to  imagery  for  situational  awareness  in  a 
format  that  ground  troops  can  easily  use  and  can  readily 


access  using  existing  Web-based  viewing  tools.  (For 
Harry  Potter  fans,  this  would  provide  the  equivalent 
capability  to  the  magical  “Marauders  Map”).  For  civil 
applications,  the  same  type  of  situational  awareness 
would  benefit  disaster  recover  efforts.  For  example,  lives 
could  have  been  saved  in  the  Katrina  disaster  with  the 
ability  to  provide  responders  precise  geo-registered 
positions  of  victims  with  accurate  images  depicting  their 
condition  and  environment.  Forest  fire  hot  zones  could  be 
accurately  identified  and  located  quickly  for  special 
treatment.  Homeland  security,  police  standoffs  or  hostage 
situations  could  be  managed  more  efficiently  with  a  better 
outcome  because  of  real  time  precision  image  availability 
that  precisely  locates  all  parties. 

For  commercial  location  servers,  near  real-time  imagery 
would  address  the  desire  of  their  customers  for  more 
current  and  accurate  data.  This  would  open  up  new 
market  and  service  opportunities  providing  current 
information  on  road  conditions,  weather  or  even  current 
events  (Figure  16).  Reducing  the  age  of  location  server 
imagery  from  years  to  minutes  would  be  a  huge  leap 
ahead  for  this  exciting  new  technology.  The  GI-Eye  and 
GRIM  products  are  being  provided  by  NAVSYS  to 
enable  this  next  generation  advancement. 


Figure  15  Urban  Real-Time  Auto-Mosaic  Collection 
Concept  of  Operations 


Figure  16  800,000  plus  people  in  Lebanon  spelling  out 
the  word  “Truth”  in  English  and  Arabic 
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